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ABSTRACT 



A study was conducted to describe instructors' attitudes 
toward and use of assessment methods in postsecondary technical and 
occupational classrooms and laboratories. Data were collected through a 
questionnaire mailed to a random sample (n=318) of all postsecondary 
technical and occupational instructors in Ohio. Results indicated that 
instructors tended to use information from paper-and-pencil tests most often 
in making instructional decisions, followed by information from performance 
assessments and informal observations. Postsecondary instructors tended 
rarely to use information from standardized tests, portfolios and essay 
methods. Instructors neither agreed or disagreed that they faced potential 
constraints while conducting assessment activities. More than four-fifths of 
the instructors reported a positive attitude toward assessment and more than 
one-half considered themselves very competent in assessment. Marketing 
instructors used portfolios more often in making instructional decisions. Two 
program areas (marketing and business) were the primary users of essay 
methods. Instructors' attitudes toward assessment were found to explain 11 
percent of the variation in use of performance assessment and 4 percent in 
use of portfolios. The perceived competency level of instructions in 
assessment contributed to about 3 percent of the variation in use of 
portfolios. (Contains 14 references.) (KC) 
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Abstract 



This descriptive study was designed to describe instructors’ use of assessment methods in 
post-secondary technical and occupational classrooms and laboratories. Data were collected from a 
random sample of all post-secondary technical and occupational instructors within the State of Ohio. 
Results indicated instructors tend to use information from paper and pencil test the most in making 
instructional decisions. Instructors used information from performance assessments the next most 
often, which was followed by informal observations. Post-secondary instructors tended to rarely use 
information from standardized tests, portfolios, and essay methods. Instructors neither agreed or 
disagreed that they faced potential constraints while conducting assessment activities. More than 
four-fifths of the instructors reported a positive attitude toward assessment and more than one-half 
considered themselves to be very competent in assessment. It was found that instructors within the 
program area of marketing accounted for more than six percent of the variation of using portfolios in 
making instructional decisions. Two program areas (marketing and business occupations) each 
accounted for three percent of the variation in the use of essay methods in making instructional 
decisions. Instructors’ attitudes toward assessment was found to explain 1 1% of the variation in 
instructors use of performance assessment and four percent of the variation in instructors’ use of 
portfolios. The perceived competency level of instructors in assessment contributed about three 
percent of the variation in instructors’ use of portfolios. 
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TECHNICAL AND OCCUPATIONAL EDUCATION INSTRUCTORS' 

PERCEIVED USE OF STUDENT ASSESSMENT INFORMATION 
IN MAKING EDUCATIONAL DECISIONS 

Since the 1970s the public, elected officials, state agencies, and the media have become more 
interested in education. This interest has been due in part to the achievement of our students, which 
does not compare well with counterparts in other countries. Hudgins (1993) stated that although 
costs have been rising for higher education, student achievement has flattened out and sometimes 
even declined. Many individuals in post-secondary education paid little attention to the debate caused 
by this report and assumed that this was a problem of K-12 education. The public, however, did not 
see the separation between post-secondary and K-12 education and looked at this as one educational 
system (Hudgins, 1993). Because of this lack of separation, there has been an increased interest and 
sometimes demand for educational institutions to show greater accountability to suggest how much 
and what students are learning at all levels of education. Post-secondary institutions have not shown 
to legislators, or to the public, a strong and obvious correlation between investment and return 
(Hudgins). 

, Colleges and universities know how smart their students were when these students arrived on 
their campuses, but they do not know very much about what they have learned while they were there 
(Bok, 1992). Because there has been little investigation on how well post-secondary instructors 
teach or how much their students learn, deciding the effectiveness of their programs has been 
difficult. This presents a problem when trying to convince the public that education is a top priority 
of these institutions. Since this focus has not shifted, colleges and universities have continued to be 
vulnerable to the attacks on their curricula, faculty, tuition, and other issues (Bok, 1992). 

Statement of the Problem 

How technical and occupational instructors use assessment information in the classroom and 
laboratory and whether its use is effective, can play a major role in enhancing and documenting both 
instruction and learning. The literature revealed very little about assessment practices and their use 
by post-secondary instructors. Recent research conducted on the quality and effectiveness of 
general-education instructor training in measurement and assessment has implied that these 
individuals may not be acquiring the appropriate skills necessary to use assessment skills effectively. 
If this is true for general education instructors, what are the assessment skills needed by technical and 
occupational instructors? Information on the assessment skills needed and the use of assessment 
information by technical and occupational education instructors is not available. 

Purpo.se and Objectives of the Study 

The purpose of this study was to describe assessment activities, problems, attitudes, and 
perceptions of Ohio post-secondary technical and occupational education instructors. The specific 
objectives of this study were to: 

1. Describe Ohio post-secondary technical and occupational education instructors in terms of 
the following demographic characteristics: age, sex, related work experience, teaching 
experience, program area taught, and educational level. 

2. Describe Ohio post-secondary technical and occupational education instructors' perceptions 
of the use of student assessment information in making instructional decisions. 

3. Describe how Ohio post-secondary technical and occupational education instructors face 
potential constraints while conducting assessment activities. 
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4. Describe Ohio post-secondary technical and occupational education instructors’ attitudes 
toward the assessment process. 

5. Describe Ohio post-secondary technical and occupational education instructors' perceptions 
of their level of competence in the assessment process. 

6. Examine the relationships between Ohio post-secondary technical and occupational 
education instructors’ use of assessment information and: 

a. selected demographic characteristics, 

b. perceived level of competence in the assessment process, 

c. their attitudes toward assessment, and 

d. constraints to the assessment processes. 

7. Determine the proportion of variance in Ohio post-secondary technical and occupational 
education instructors' perceived use of assessment information in instructional decision- 
making that can be explained by the independent variables of attitudes toward assessment, 
competence in the assessment process, constraints to the assessment process, and selected 
demographic characteristics. 



METHODOLOGY 

The methodology used to conduct the research is described in this section. The methodology 
is organized and presented in the following sections: (a) research design, (b) population and 
sampling, (c) instrumentation, (d) data collection, and (e) data analysis. 

Research Desigi 

This descriptive-correlational study was designed to explore and describe the assessment 
practices of Ohio post-secondary technical and occupational instructors. The nature and strength of 
relationships between assessment use and instructor attitudes, instructor competence in assessment, 
constraints to assessment, and instructor characteristics were examined. The dependent variable in 
this study was instructor use of student assessment information in making instructional decisions. 
Independent variables included instructor attitudes toward assessment, instructors’ competence in 
assessment, and potential constraints faced by instructors while conducting assessment activities. 

Population and Sampling 

The target population for this study was all full-time post-secondary technical instructors in 
the State of Ohio during the spring of 1996. A random sample of 318 individuals was drawn from the 
population as recommended by Krejcie and Morgan (1970) with a 95% confidence level of obtaining a 
representative sample of the population. 



Instrumentation 

The five-part survey used for this research was adapted from a survey used in a similar 
research study (Kershaw, 1993). The dependent variable, instructor use of assessment information, 
was measured in Part I of the instrument. Instructors were asked to indicate the extent to which they 
use information from six types of assessment methods in addressing nine different instructional 
decisions. The six types of assessment methods used in the study included: objective paper and 
pencil methods, standardized test scores, performance assessments, informal observations, 
portfolios, and essay methods. The participants in this study were asked to indicate the extent to 
which they use results from each of the six assessment methods in making nine different types of 




2 



5 



instructional decisions. The instructional decisions addressed were: planning for instruction, 
diagnosing student weaknesses, monitoring student progress toward course objectives, motivating 
students to learn, evaluating the effectiveness of instruction, evaluating the instructional materials 
used, encouraging students to assess their own work, assigning grades, and grouping students for 
instructional activities. A five-point Likert-type scale ranging fi-om “never use” to “always use” was 
used. 

Part II of the questionnaire was designed for measuring the first independent variable, 
competence in assessment. The respondents were asked to indicate the perceived level of their 
competence in the assessment process. The competency statements used for this section were based 
upon “Standards for Teacher Competence in Educational Assessment of Students” (American 
Federation of Teachers, National Council on Measurement in Education, &, National Education 
Association, 1990). A five-point Likert-type scale was used which ranged fi’om “not competent” to 
“extremely competent.” 

Instructors’ attitudes toward assessment were measured in Part in of the instrument using a 
semantic differential scale. This scale consisted of eight bi-polar adjectives that represent the 
concept of assessment. A seven-point scale was used for each pair of adjectives. Respondents were 
asked to place a check mark between the adjectives at the point that best indicated how they would 
describe their attitude toward the overall assessment process. 

Part rV of the instrument was designed to measure the perceived constraints that instructors 
may face during their assessment activities. A five-point Likert-type scale was used that ranged 
fi’om “strongly disagree” to “strongly agree”. Participants were asked to indicate their level of 
agreement or disagreement with nine constraint statements. 

The personal characteristics of each instructor were measured in Part V of the questionnaire. 
Respondents were asked to provide personal information regarding the characteristics of sex, 
teaching experience, related work experience, teaching area, level of education and present age. 

A panel of experts was used to establish content and face validity for the survey instrument. 
The panel of experts was used to ensure that the instrument had clearly defined statements and 
questions, made suggested changes to statements and questions, offered suggestions for the addition 
or deletion of statements and questions, and made comments on the format and general appearance of 
the survey instrument. 

After the instrument was revised, it was pilot tested for reliability using a group of 
respondents from a community college outside the State of Ohio. A test-retest method was also used 
to test the reliability of the instrument over a period of time. The time interval between the test and 
retest was two weeks. The results from test #1 and test #2 were compared for percent agreement 
with values that ranged from .85 to .92 for each item. Measures of internal consistency were 
calculated from the data obtained from the first test of the test/retest method. Cronsbach’s Alpha 
ranged from .77 to .94 for each of the six sections of Part I, .85 for the 21 summed items in Part II, 

.81 for the semantic differential in Part IH, and .54 for the sum of the nine items in Part IV. 

Data Collection 

The data for this study were collected by a mail questionnaire. This data collection consisted 
of an initial mailing of questionnaires being sent out and approximately one week later a post card 
reminder was sent to all participants in the study. This post card reminded participants of the 
importance of the study and the importance of each response. It further thanked the respondents and 
asked the non-respondents to take the time to complete and return the questionnaire. Approximately 
two weeks after the first mailing a second packet was mailed to all non-respondents. The final 
correspondence with the participants included a second post card reminder sent to all non- 
respondents approximately one week after the second mailing was sent. Questionnaires were 
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received from 170 instructors that represented an overall response rate of 53%. Of the 170 surveys 
returned, 93% (158) were useable. This represented an overall useable response rate of slightly less 
than 50%. 

A 10% random sample of non-respondents was selected and contacted by telephone. 
However, to obtain 10% responses, it was necessary to contact more than 20% of the non- 
respondents. Differences between non-respondents and respondents on each part of the 
questionnaire were compared using a 1-test. The significance level of alpha for this test was set at 
.05. No significant differences were found between the two groups for each part of the questionnaire. 
However, an over sampling of the non-respondents was conducted to obtain this information, 
therefore the results of this study caimot be generalized beyond the respondents. 

Data Analysis 

Descriptive statistics provided measmes of central tendency and variability for organizing, 
summarizing and analyzing differences and likenesses between groups. Pearson’s i coefficient, point 
bi-serial correlation coefficients, and Cramer’s V were used to summarize the magnitude and 
direction of the relationship between variables. The Davis (1971) conventions were used to describe 
the measmes of association (Table 1). 

Table 1 

Conventions Used to Describe Measures of Association 



Coefficient 


Description 


.70 or higher 


Very Strong Association 


.50 to .69 


Substantial Association 


.30 to .49 


Moderate Association 


.10 to .29 


Low Association 


.01 to .09 


Negligible Association 



Somce: Davis, (1971) 



Semi-partial multiple regression was used to calculate the unique relationships between 
independent variable sets and the dependent variable, use of assessment information in making 
instructional decisions. The stepwise entry method of independent variables into the multiple 
regression models was used based on the exploratory nature of the study, the fact that interval, 
ordinal, and nominal variables were investigated, and the lack of theory to guide the order in which 
variables were entered into the equation. 



FINDINGS 

This section reports findings related to the following: (a) characteristics of the respondents, 
(b) instructors’ use of assessment information, (c) instructors’ perceived competence in assessment, 
(d) instructors’ attitudes toward assessment, (e) instructor perceived constraints to assessment, (f) 
relationships between dependent and independent variables, and (g) multiple regression models. 

Demographic and Background Characteristics of the Sample 

This study examined six demographic and background characteristics for the sample that 
included age, sex, years of teaching experience, years of work experience, program area, and level of 
education. From the variable of age it was found that 44.2% of the sample was in an age group that 
ranged from 40-49 years (Table 2). Slightly more than 27% of the respondents were between 50-59, 
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and 16.2% were between 30-39 years of age. Instructors ranged in age from 25 to 67 with a mean 
age of 47.03. Gender data revealed that females comprised 57% and males 43% of the sample 
(Table 2). 

Table 2 

Age and Sex of Instructors (n=158) 



Age 



Sex 



Characteristics 


f 


% 


Under 30 


3 


1.9 


30-39 


25 


16.2 


40-49 


68 


44.2 


50-59 


42 


27.3 


60 and over 


16 


10.4 


Total 


154" 


100.0 



Female 


90 


57.0 


Male 


68 


43.0 


Total 


158 


100.0 



Mean = 47.03 
SD = 9.4 



“Information was missing for four respondents on age. 

Years of teaching experience ranged from one to 38 years with the mean being 15 years of 
experience (Table 3). The largest group of instructors (23.1%) taught for 16-20 years while the 
smallest group (2.6%) taught for more than 3 1 years. In terms of related work experience, 12.3% 
had 16-20 years of related work experience before or concurrent to teaching, 18.8% had 1 1-15 years, 
and 24.7% had 6-10 years (Table 3). Slightly more than 34% had 0-5 years of related work 
experience. 
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Table 3 



Instructors’ Years of Related Work and Teaching Experience (n = 158) 



Characteristics f % 



Years of Related Work Experience 



0-5 


53 


34.4 


6-10 


38 


24.7 


11-15 


29 


18.8 


16-20 


19 


12.3 


21-25 


5 


3.3 


26-30 


6 


3.9 


31-35 


4 


2.6 


Total 


154" 


100.0 


rs of Teaching Experience 


1-5 


20 


12.8 


6-10 


32 


20.5 


11-15 


27 


17.3 


16-20 


36 


23.1 


21-25 


19 


12.2 


26-30 


18 


11.5 


3 1 or more 


4 


2.6 


Total 


156” 


100.0 



Mean = 10.96 
SD = 7.92 



Mean = 15.49 
SD = 8.34 



‘Information was not available from four respondents on years of related work experience. 
’’Information was not available from two respondents on years of teaching experience. 

Health occupations comprised the largest group in the study with 38.5% of all instructors 
(Table 4). Business occupations also consisted of a large portion of instructors within the study 
(26.3%), and was followed by engineering at 12.8%. Agriculture, family and consumer sciences and 
human services contributed less than two percent each. Four-fifths of the respondents had 
completed a graduate degree (80%) (Table 5). Slightly less than 10% had completed some graduate 
work, and less than six percent had a bachelors degree. 
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Program Area of Instructors (n=158) 



Program Area 


f 


% 


Agriculture 


3 


1.9 


Business Occupations 


41 


26.3 


Marketing 


5 


3.2 


Family and Consumer Sciences 


2 


1.3 


Trade and Industrial 


10 


6.4 


Health Occupations 


60 


38.5 


Engineering 


20 


12.8 


Human Services 


2 


1.3 


Other 


13 


8.3 


Total 


156“ 


100.0 


“Information was not available from two respondents. 
Table 5 






Instructors Level of Education (n=l 58) 


Level of Education 


f 


% 


Associates Degree 


5 


3.2 


Bachelors Degree 


9 


5.8 


Some Graduate Work 


15 


9.7 


Masters Degree 


106 


68.4 


Doctorate 


18 


11.6 


Other 


2 


1.3 


Total 


155“ 


100.0 



“Information was not available from three respondents. 



Use of Assessment Information 

Table 6 displays the results of instructors’ use of assessment information. Instructors rated 
assessment information provided from objective paper and pencil methods (M=3.92) as more useful 
in making instructional decisions than information from the other five assessment methods. 
Performance assessment was rated second highest by instructors for providing information in making 
decisions (M=3.78). Information derived from informal observations was rated third (M=3.64\ 
essay items were fourth (M=2.49). and standardized test scores rated fifth (M=l .90). Portfolios 
provided information deemed of the least use by instructor in making educational decisions 
(M=1.78). 



Use of Assessment Information Generated From Six Assessment Methods (n=158) 
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6. Evaluating instructional material 

7. Encouraging students to assess their own work 

8. Assigning grades 

9. Grouping students for instructional activities 
‘‘Bimodal item with modes of 4 and 5 



Potential Constraints to Assessment 



Table 7 depicts the summed data gathered from instructors on potential constraints they 
faced while conducting assessment activities. Over one-half (55.1%) of the instructors neither 
agreed nor disagreed that they faced potential constraints while conducting their assessment 
activities. Slightly more than 42% of instructors agreed that they faced potential constraints when 
conducting assessment activities, while slightly more than one percent disagreed potential constraints 
affected their assessment activities. 

Table 7 

Overall Constraints to Assessment Scores (n = 158) 



Scale Value 


Constraint Score 


f 


% 




Strongly Disagree 


9-13 


0 


.00 




Disagree 


14-22 


2 


1.27 


Mean = 30.82 


Neither Agree or Disagree 


23-31 


87 


55.05 


Agree 


32-40 


67 


42.41 


SD = 3.97 


Strongly Agree 


41 -45 


2 


1.27 





Note. Missing data was recoded to the mean. 



Instructors’ Attitude Toward Assessment 



Attitudinal measure of post-secondary instructors was made by summing the eight responses 
on the semantic differential scale. Post-secondary instructors reported having a more positive 
(81.1%) than negative (5%) attitude toward assessments (Table 8). The remaining instructors 
(13.9%) in this study reported being neutral in their attitude toward assessment. 

Table 8 

Instructors’ Summated Attitude Scores Towards Assessment (n = 158) 



Summed Attitude Scores 



f 



8-11 


0 


0.0 


12-19 


1 


0.6 


20-27 


7 


4.4 


28-35 


22 


13.9 


36-43 


47 


29.8 


44-52 


62 


39.2 


53-56 


19 


12.1 



Mean = 42.65 
SD = 8.19 



Note. Missing data was recoded to the mean. 

Perceived Level of Competence in Assessment 

A measure of perceived competence in assessment was calculated by summing post- 
secondary instructor responses to 21 competency statements (Table 9). The summed scores across 
the 21 statements indicated that slightly less than six percent of technical and occupational 
instructors considered themselves to be extremely competent in the overall assessment process, 
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51.9% to be very competent, and 42.4% to be moderately competent. None of these instructors 
considered themselves to be slightly competent or not competent in assessment. 

Table 9 



Summed Instructor Competency Scores (n = 158) 



Scale Value 


Summed Score 


f 


% 




Not Competent 


21-31 


0 


0.0 




Slightly Competent 


32-52 


0 


0.0 


Mean = 76.5 


Moderately Competent 


53-73 


67 


42.4 


SD = 10.76 


Very Competent 


74-94 


82 


51.9 


Extremely Competent 


95 - 105 


9 


5.7 





Note. Missing data was recoded to the mean. 



Relationships Between Dependent and Independent Variables 

Potential constraints faced by instructors when conducting assessment activities was found 
to have a low association with the use of standardized test scores (£=.25) and performance 
assessment (£=.20) (Table 10). Attitude toward assessment had a moderate positive association 
(£=.33) with the use of performance assessment. Program area had a substantial association (£^=.58) 
with the use of essays as an assessment method, and competence in assessment had a low positive 
association (£=.18) with instructor use of information from portfolios as an assessment method. 
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Intercorrelations Between Independent Variables and Assessment Methods (n=158) 
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Regression of Dependent Variable Upon the Independent Variable 



It was found that instruetors within different program areas plaeed more weight upon the use 
of portfolios as an assessment tool (Table 11). The program area of marketing aeeounted for more 
than six percent of the variation of using portfolios in making instruetional deeisions. Two program 
areas (marketing and business oeeupations) eaeh aeeounted for slightly more than three pereent of 
the variation in the use of essay methods in making instruetional deeisions (Table 12). 

Table 11 

Semi-Partial Regression of Use of Portfolios On All Independent Variables (n = 158)(Stepwise 
Entry) 



Variables 


sR' 


b 


t 


D 


Marketing 

(Program Area)® 


.065 


1.53 


3.49 


<.001 


Potential Constraints 


.029 


-.581 


-3.30 


<.001 


Attitude 


.043 


.184 


2.38 


.018 


Competence 


.033 


.367 


2.46 


.015 


(Constant) 


1.42 








Standard error = .935 
R^=.17 

Adjusted R^= .15 

For model: £ = 7.37, p <.001 

“Multichotomous variables were dummy coded: Marketing 


= 1 and Other = 0 






Table 12 










Semi-Partial Regression of Use of Essay Methods On All Independent Variables (n = 


158)(Stepwise 


Entry) 








Variables 


sR' 


b 


t 


D 


Marketing 

(Program Area)® 


.033 


1.26 


2.51 


.013 


Business Occupations 
(Program Area)® 


.031 


.448 


2.23 


.027 


(Constant) 


2.35 









Standard error = 1.077 
R^=.064 

Adjusted = .052 

For model: F = 5.08, g <.001 

“Multichotomous variables were dummy coded: Marketing = 1 and Other = 0. Business Occupations = 1 and 
Other = 0 
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Instructors’ attitudes toward assessment had a moderate positive association (r=.33) with the 
use of performance assessment and was found to explain 11.2% of the variation in instructors use of 
the assessment method (Table 13). Attitude toward assessment was also found to explain 4.3% of 
the variation in instructors’ use of portfolios as an assessment method (Table 11). 



Table 13 



Semi-Partial Regression of Use of Performance Assessment On All Independent Variables 
(n = 158)( Stepwise Entry) 



Variable 


sR" 


h 


I 


E 


Attitude 


.112 


.248 


4.33 


<.001 


(Constant) 


2.45 









Standard error = .742 



R'=.112 

Adjusted .106 

For model: £ = 18.73, g <.001 



Potential constraints faced by instructors during the assessment process explained 2.8% of 
the variation in the instructor’s use of standardized test scores while the program area of Health 
Occupations explained 9.2% of the variation (Table 14). There was not a statistically significant 
association between attitude toward assessment and use of portfolios, yet when it was entered into a 
regression model with three other variables (marketing, constraints, competence) it explained 4.3% 
of the variation in instructors’ use of portfolios as an assessment method (Table 11). 



Table 14 

Semjjgartial Regression of Use of Standardized Test Scores On All Independent Variables 
(n = 158)(Stepwise Entry) 



Variables 


sR^ 


h 


I 


E 


Health Occupations 
(Program Area)® 


.092 


.405 


3.14 


<.001 


Potential Constraints 


.028 


.555 


2.19 


.03 


(Constant) 


.329 









Standard error =1.01 
R'=.120 

Adjusted R^= .108 

For model: £ = 10.11, g <-001 

“Multichotomous variables were dummy coded: Health = 1 and Other = 0 
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Competency level was found to contribute only 3.3% of the variation in instructors’ use of 
portfolios (Table 1 1). While competence accounted for a small percent of variation in instructors’ 
use of portfolios as an assessment method, it did not explain a significant amount of variance in the 
use of objective paper and pencil methods, standardized test scores, performance assessments, 
informal observations, and essay assessment methods. 

CONCLUSIONS AND RECOMMENDATIONS 

This descriptive study was designed to describe instructors’ use of assessment methods in 
post-secondary technical and occupational classrooms and laboratories. The study further 
investigated the relationships between use of student assessment information and characteristics of 
post-secondary technical and occupational instructors in the State of Ohio. 

Conclusions 

This section contains the conclusions reached based upon the findings fi-om this study. As a 
reminder, an attempt was made to control for non-response error by comparing non-respondents to 
respondent. Over sampling of the non-respondents was necessary to obtain a sufficient number of 
respondents to make a comparison using a Hest. Therefore, the results of this study cannot be 
generalized to the population. 

Use of Assessment Information 

Respondents reported using objective paper and pencil methods the most for student 
assessments. Since technical and occupational instructors teach subjects that involve many student 
performance activities, it was surprising that objective paper and pencil methods of assessment were 
used more than performance assessments. In previous research, Kershaw (1993) found that 
secondary vocational teachers tended to place more emphasis upon the use of performance 
assessments than on other assessment methods. 

Standardized test scores were found of little use in making instructional decisions by the 
respondents. This tends to indicates that instructors use of standardized testing continues to be little 
by instructors in making educational decisions. 

Portfolios were found the least used source of student assessment information for decision- 
making. Although this assessment method continues to be promoted as an effective assessment 
method within education, instructors who responded tended not to use the information generated 
fi’om portfolios. 

Potential Constraints to Assessment 

Post-secondary technical and occupational instructors within this study were split between 
agreeing and neither agreeing nor disagreeing regarding the potential constraints to the assessment 
process. The results of this study were similar to those findings reported by Gullickson (1984). 
Kershaw (1993) also found similar results in his study of secondary vocational teachers. In addition 
to the predominately neutral findings, other studies found that limited time for planning was the 
constraint that received consistent higher levels of agreement. 

Instructors’ Attitude Toward Assessment 

For the most part, the respondents had positive attitudes toward assessment and agreed with 
those reported by Green (1990) where attitudes of both preservice and experienced teachers toward 
classroom assessment were positive. Additionally, Kershaw (1993) found that secondary teachers 
tended to have a positive attitude toward assessment. 
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Perceived Level of Competence in Assessment 

Technical and occupational educators in this study reported that they perceived themselves 
to be moderately to extremely competent in the assessment process. These results were in line with 
findings fi-om Gullickson and Hopkins (1987) that described teachers as being comfortable in their 
knowledge and abilities related to the assessment process. Dorr-Bremme (1983) also concluded that 
teachers perceived their use of assessment techniques as accurately measuring the effects of their 
instruction. 

Results fi'om this study, in the area of the competency level of instructors in assessment, 
conflicted with other research. Previous research found that teachers were lacking the necessary 
skills in assessment selection, development, and use (Newman & Stallings, 1982; Carter, 1984; 

Hills, 1991). In addition, Gullickson and Hopkins (1987) and Schafer and Lissitz (1987) supported 
the position that preservice courses have not been adequate in developing the level of desired 
assessment skills in teachers. 

From previous research, there has been evidence to suggest that teachers may tend to 
exaggerate their level of competence in assessment when data is obtained in a self-reported manner. 
Marso and Pigge (1989) found that teachers, principals, and supervisors do not agree on the 
proficiency level of teachers’ test construction and test planning proficiencies. Teachers tended to 
rate themselves as much more proficient in assessment than did their supervisors or principals. 
Findings reported by Green and Williams (1989) indicated that teachers with less training in tests 
and measurements perceived themselves to be more knowledgeable about interpreting standardized 
test scores than teachers with more training in this area. If the instructors’ perceived competence 
levels can be interpreted as being exaggerated, then the findings that 42.4% of instructors reported a 
moderate level of competence in assessment becomes more important. The higher levels of perceived 
competence, the small significant relationship between perceived competence and the use of 
portfolios, and the past research that identified the deficiencies of teacher assessment skills suggest 
the potential need for the upgrading of technical and occupational education instructor competence in 
assessment practices. 

Relationships Between Dependent and Independent Variables 

Program areas had a substantial association with the use of essays as an assessment method. 
This finding tended to indicate that respondents fi-om marketing and business occupations used essay 
methods more than their counterparts in other technical and occupational programs. Competence in 
assessment had a low positive association with instructor use of information fiom portfolios as an 
assessment method indicating that respondents who perceived themselves as more competent in 
assessment tended to be more likely to use portfolios for assessment. 

A moderate relationship (r=.33) was present between attitude and use of performance 
assessment, which conflicts with some previous research. Green and Stager (1986) concluded that 
the relationship between attitude and assessment use was of no practical importance. Kershaw 
(1993) also concluded that the relationship between attitude and assessment use was of little 
importance. The results of this study indicated that although attitude may not be important in the use 
of all the different types of assessment methods, it was of some importance when using performance 
assessment. 

Regression of Dependent Variable Upon the Independent Variables 

This study failed to demonstrate that instructor characteristics accounted for much of the 
variation in their use of assessment information. Age, sex, years of teaching experience, years of 
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related work experience, and educational level did not account for a significant proportion of the 
variation in the respondents’ use of any assessment method. 

Though there was not a statistically significant association between program area and 
standardized test scores, it was found that the health occupations program area explained nine 
percent of the variation in the instructors’ use of standardized test scores. This may have been due to 
the use of standardized test scores as an entry and exit requirement for students in many health 
occupation programs. 

Program area was found to have a substantial association with the use of essay methods for 
making educational decisions. Two program areas (marketing and business occupations) each 
accounted for three percent of the variation in the use of essay methods in making instructional 
decisions. This indicated that essay methods may be used more by instructors in marketing and 
business occupation programs than instructors in other program areas. 

This study was unable to support the claim that potential constraints faced by instructors 
while conducting assessment activities played a major part in explaining variation in instructor use of 
assessment methods. The use of two assessment methods (portfolios and performance assessments) 
were shown to be related to constraints faced by instructors while conducting assessment activities, 
but a very small proportion of variance was explained by these constraints. 

Competence accounted for a small percent of variation in instructors’ use of portfolios as an 
assessment method, however, it did not explain a significant amount of variance in the use of 
objective paper and pencil methods, standardized test scores, performance assessments, informal 
observations, and essay assessment methods. These findings were slightly different fi-om what 
Kershaw (1993) found. Kershaw found that competence in assessment contributed a significant, yet 
small portion of variance in teachers’ use of objective paper and pencil methods, use of performance 
assessment and use of informal observation. 

Recommendations 

Recommendations fi’om the results of this study will be made for practice and future 
research. Based upon the review of literature, the findings of this study, and the previous conclusions, 
the following recommendations are proposed. 

Recommendations for Research 

1 . Because this study was able to explain only a small amount of variance in post-secondary 
technical and occupational instructors’ use of assessment methods, it is recommended that 
each of these methods be studied independently. Also, it is suggested that qualitative 
research be conducted on each of the assessment methods to help identify other variables 
that may contribute additional variance in explaining the use of assessment information in 
making educational decisions. Open-ended responses could be collected regarding instructor 
attitude toward assessment, competence in assessment, and potential constraints in the 
assessment process. 

2. Since attitude toward the overall process contributed little to understanding the use of 
assessment by instructors, it is recommended that future research relate this attitude toward 
the specific assessment methods. What are instructors’ attitudes toward objective paper and 
pencil assessment methods, standardized testing, performance assessment, informal 
observations, portfolios, and essay methods? Information gathered on the attitudes toward 
each assessment method may explain more about the use of these methods by educators. 

3. Individual competence in assessment was rated rather high by instructors in this study, yet it 
only explained a small proportion of variation in the use of assessment methods. It is 
recommended that future research investigate the level of competence of the instructors in 
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each assessment method. Information generated on the level of competence with each 
assessment method may find different levels of competence within each assessment method. 
Research such as this may also identify strengths and weaknesses that instructors have 
related to the assessment process. This data would be valuable to in-service and pre-service 
teacher education programs in updating their courses and workshops. 

4. Instructors rated their level of competence on the high side in this study, yet it explained 
little of the variation in the use of assessment methods. This may indicate that instructors 
overstated their competence level. For future research on the level of instructors’ 
competence, it is suggested that self-reported competence levels be compared with observed 
instructors’ competence level. In addition, since competence in certain areas has been foimd 
to be related to education within that area, it is recommended that research be imdertaken to 
collect information at the different colleges and universities to learn how many of the courses 
in the pre-service technical and occupational education curriculum concentrate on 
assessment methods and what assessment methods are emphasized within these courses. 

5. To provide technical and occupational instructors with ideas on how the different types of 
assessment can be used within their specific programs, it is recommended that research be 
conducted to investigate and collect examples of how instructors (within the different 
program areas) use the different assessment methods. If instructors can see specific ways 
that these assessment methods are used, they may be better able to carry out these 
assessment techniques within their individual programs. 

6. Recent emphasis on the development of national skill standards will have an impact upon 
assessment in technical and occupational programs. Future research should investigate how 
and whether these standards are being used as an assessment tool. 

7. It is recommended that the findings fi-om this research be compared with what is being 
taught in pre-service and in-service teacher education programs. It may be that instructors 
are only assessing students with methods with which they are familiar. 

8. The general population has started calling for public educational institutions to be more 
accoimtable. One measure of accoimtability is the level of proficiency of the students. 
Students tested with a variety of assessment methods will provide more evidence to the 
public that these students are learning what they are supposed to learn. Consequently, 
instructors need to be able to use different kinds of assessment methods. 

Recommendations for Practice 

1. Findings fi-om this study indicate that certain demographic characteristics have little bearing 
on the use of different assessment methods. This suggests that either assessment practices 
change very little as an instructor gains instructional and educational experiences or that 
instructor assessment strategies and preferences developed early in their careers are changed 
very little over a period of time. Therefore, it is recommended that pre-service and in-service 
educators provide a curriculum that includes different assessment methods and how these 
methods can be used in the different program areas. Post-secondary educational institutions 
should provide some means of recognizing those instructors that use different assessment 
methods within their individualized programs. 

2. Post-secondary technical and occupational instructors within this study were not using 
standardized test results. Such scores may be helpful to instructors in determining strengths 
and weakness of their students. Instructors within the different program areas may need 
further education about how they may use standardized test scores to enhance their programs 
and student achievement. 
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3. It is recommended that technical and occupational instructors continue to use performance 
assessments within their programs. This assessment method continues to an appropriate 
method for measuring learning performance-based environments. 

4. Data gathered from this study indicated that the different types of assessment methods are 
not evenly used. This could be due to the lack of understanding by instructors of other 
assessment techniques. The different assessment methods may be prime candidates for 
topics of in-service opportunities for instructors. 

5. Student achievement levels can vary by the different ways they are assessed, so it is 
important students are provided the opportunity to demonstrate their knowledge and skill 
level in a variety of ways. Within the field of education, certain assessment methods tend to 
be used more than other assessment methods. Instructors should determine how and whether 
these assessment methods can be used within their program areas, but they should not shift 
all of their assessment activities to just one method. 

6. There is a saying that “we teach how we are taught”. If this is true about teaching, then it 
may also be appropriate for assessment. University faculty, in pre-service teacher education 
progr ams , should use different types of assessment methods within their classes to serve as 
examples for their students. 



REFERENCES 

American Federation of Teachers, National Council on Measurement in Education, and the 
National Education Association. (1990). Standards for teacher competence in educational 
assessment of students . Washington, DC: Authors. (ERIC Document Reproduction Service No. ED 
323 186). 

Bok, D. (1992). Reclaiming the public trust. Change. 24 (4). 12-19. 

Carter, K. (1984). Do teachers understand principles for writing tests? Journal of Teacher 
Education. 35( 6\ 57-60. 

Davis, J. A. (1971). Elementary survey analysis . Englewood, NJ: Prentice-Hall. 

Green, K. E. (1990, April). Differences J^etween teachers and students in opinions about 
testing and test use . A Paper presented at the annual meeting of the National Council on 
Measurement in Education, Boston, MA. (ERIC Document Reproduction Service No. ED 318 775). 

Green, K. E., & Stager, S. F. (1986). Measuring attitudes of teachers toward testing. 
Mea.surement and Evaluation in Counseling and Development. 19( 3). 141-150. 

Green, K. E., & Williams, E. J. (1989). Standardized test use by classroom teachere: 

Effects of training and grade level taught. Paper presented at the annual meeting of the National 
Council on Measurement in Education, San Francisco, CA. (ERIC Document Reproduction Service 
No. ED 306 252). 

Hills, J. R. (1991). Apathy concerning grading and testing. Phi Delta Kappan. 72 (7). 540- 

545. 

Hudgins, J. L. (1993). Institutional effectiveness: A maturing movement. Where do we go 
from here? Paper presented at the Summer Institute of the Community College Consortium, 
Madison, IL. (ERIC Document Reproduction Service No. ED 358 891). 




18 



23 



Kershaw IV, I. (1993). Ohio vocational education teachers’ perceived use of student 
assessment information in educational decision making . Unpublished doctoral dissertation, The 
Ohio State University, Columbus. 

Marso, R. N., «& Pigge, F. L. (1989). The status of classroom teacher’s test construction 
proficiencies: Assessments by teachers, principals, and supervisors validated by ^alvses of actual 
teacher-made tests. Paper presented at the annual meeting of the National Council of Measurement in 
Education, San Francisco, CA. (ERIC Document Reproduction Service No. ED 306 283). 

National Commission on Excellence in Education. (1983). A nation at risk: The imperative 
for educational reform. The Elementary School Journal. 84 (2). 113-130. 

Nevraian, D. C. & Stallings, W. M. (1982, March). Teacher competency in classroom 
testing, measurement preparation, and classroom testing practices . Paper presented at the annual 
meeting of the National Council on Measurement in Education, New York, NY. (ERIC Document 
Reproduction Service No. ED 220 491). 

Schafer, W. D., & Lissitz, R. W. (1987). Measurement training for school personnel: 
Recommendations and reality. Journal of Teacher Education. 38 (3). 57-63. 




19 



24 




O /^OO V 



U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 



ERIC 



REPRODUCTION RELEASE 

(Specific Document) 



I. DOCUMENT IDENTIFICATION: 




II. REPRODUCTION RELEASE: 

In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in the 
monthly abstract journal of the ERIC system, Resources in Educetion (RIE), are usually made available to users in microfiche, reproduced paper copy, 
and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, if 
reproduction release is granted, one of the following notices is affixed to the document. 

If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottom 
of the page. 



The sample sticker shown below will be 
affixed to an Level 1 documents 



Level 1 



The sample sticker shown below will be 
affixed to ail Level 2A documents 



The sample sticker shown below will be 
affixed to all Level 26 documents 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE. AND IN ELECTRONIC MEDIA 
FOR ERIC COLLECTION SUBSCRIBERS ONLY. 
HAS BEEN GRANTED BY 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE ONLY HAS BEEN GRANTED BY 

A® 












TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

1 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

2A 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

2B 



Level 2A 



Level 2B 








Check here for Level 1 release, permitting reproduction 
and dissemination in microfiche or other ERIC archival 
media (e.g., electronic) and paper copy. 



Check here for Level 2A release, pennitting reproduction 
and dissemination In microfiche and In electronic media 
for ERIC archival collection subscribers only 



Check here for Level 26 release, permitting 
reproduction and dissemination in microfiche only 



Documents will be processed as Indicated provided reproduction quality permits. 

If permission to reproduce is granted, but no box is checked, documents vrill be processed at Level 1 . 



Sign 
here,-^ 
) nlease 



/ hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate this document 
as indicated above. Reproduction from the ERIC microfiche or electronic media by persons other than ERIC employees and its system 
contractors requires permission from the copyright holder. Exception is made for non-profit reproduction by libraries and other sen/ice agencies 
to satisfy information needs of educators in response to discrete inquiries. 




Pwted Name/Position/Title: a / / Ci 1 


OrganizatiorVAddress:/ / e\ ( ^ 

d^luQrsfry^ |/0( isclK K&<KPU/aV 




"“7/WJI5- V05C 


E-Mail Address: /\ ii / / 





'f^cir^AJsiSjJr ^ Pi'll - ^ rc (fi 



(over) 



ill. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE): 

If permission to reproduce is not granted to ERiC, or, if you wish ERiC to cite the availability of the document from another source, please 
provide the foiiowing infonnation regarding the avaiiability of the document. (ERiC will not announce a document unless it is publicly 
available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are significantiy more 
stringent for documents that cannot be made available through EDRS.) 



Publisher/Distributor: 



Address: 



Price: 



IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER: 

If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name and 
address: 




V. WHERE TO SEND THIS FORM: 



Send this form to the following ERIC Clearinghouse: 

Acquisitions Coordinator 

ERIC Clearinghouse on Adult, Career, and Vocational Education 
Center on Education and Training for Employment 
1900 Kenny Road 
Columbus, OH 43210-1090 



However, if solicited by the ERIC Facility, or if making an unsolicited contribution to ERIC, return this form (and the document being 
contributed) to: . 



E KlC fl (Rev. 9/97) 

PREVIOUS VERSIONS OF THIS FORM ARE OBSOLETE. 



